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DETAILED ACTION 

1 . Claims 1 -6 and 8-20 are pending in this application. 

Claim Objections 

2. Claim 6 is objected to under 37 CFR 1 .75(c), as being of improper dependent 
form for failing to further limit the subject matter of a previous claim. Applicant is 
required to cancel the claim(s), or amend the claim(s) to place the claim(s) in proper 
dependent form, or rewrite the claim(s) in independent form. Claim 1 now contains the 
subject matter of Claim 6. 



Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

4. This application currently names joint inventors. In considering patentability of 
the claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of 
the various claims was commonly owned at the time any inventions covered therein 
were made absent any evidence to the contrary. Applicant is advised of the obligation 
under 37 CFR 1 .56 to point out the inventor and invention dates of each claim that was 
not commonly owned at the time a later invention was made in order for the examiner to 
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consider the applicability of 35 U.S.C. 103(c) and potential 35 U.S.C. 102(e), (f) or (g) 
prior art under 35 U.S.C. 103(a). 



5. Claims 1-6, 8-12,14-16 and 20 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Snook, US Patent 6400378, and further in view of Dufaux, US Patent 
6711587. 

As in Claims 1 and 15, Snook teaches a media handling system in which 
candidate video sequences are displayed on a display screen in schematic form for 
selection by a user (Fig. 3 and corresponding text) and method (Fig. 3 and 
corresponding text), the system and method comprising: a display screen configured to 
display representations of the candidate video sequences for selection by a user (ref. 
120), each representation including one or more images derived from the respective 
video sequences (Col. 2, lines 64-66); and a user control means for defining a set of 
one or more of the video sequences (Col. 2, line 55-Col. 3, line 1). While Snook 
teaches displaying video sequences, represented by a keyframe thumbnail, that may be 
for selected by a user to define a set of one or more video sequences, they fail to show 
the means for detecting human faces in the candidate video sequences to represent the 
candidate video sequences as a keyframe as recited in the claims. In the same field of 
the invention, Dufaux teaches a media handling system similar to that of Snook. In 
addition, Dufaux further teaches means for detecting human faces in the candidate 
video sequences (Fig. 5, ref. 500 and corresponding text); and assigning them as the 
key frame representations of the candidate video sequences (Col. 4, lines 4-7). It would 
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have been obvious to one of ordinary skill in the art, having the teachings of Snook and 
Dufaux before him at the time the invention was made, to modify displaying video 
sequences, represented by a keyframe thumbnail, that may be for selected by a user to 
define a set of one or more video sequences taught by Snook to include the means for 
detecting human faces in the candidate video sequences and assigning them as the key 
frame representations of the candidate video sequences of Dufaux, in order to obtain 
means for detecting human faces in candidate video sequences, displaying video 
sequences, represented by an image representing human faces derived from the 
respective video sequences, that may be for selected by a user to define a set of one or 
more video sequences. One would have been motivated to make such a combination 
because a way for the system to select a more relevant, user friendly representative 
image would have been obtained, as taught by Dufaux. 

Snook fails to explicitly teach detecting human faces derived from the respective 
video sequences and displaying the one with the highest probability of containing a face 
in the group. Dufaux teaches the detecting means is operable to detect a probability of 
a human face being present in each field or frame of the video sequences (Col. 1 , lines 
55-57); each displayed representation of a candidate video sequence including one or 
more images representing human faces which have the highest probability levels 
amongst the respective video sequences (Col. 12, line 4 et seq.). It would have been 
obvious to one of ordinary skill in the art, having the teachings of Snook and Dufaux 
before him at the time the invention was made, to modify displaying video sequences, 
represented by a keyframe thumbnail, that may be for selected by a user to define a set 
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of one or more video sequences and a further ordered representation of a group of at 
least a subset of the video sequences forming the output media product taught by 
Snook to include the detecting human faces derived from the respective video 
sequences and displaying the one with the highest probability of containing a face in the 
group of Dufaux, in order to obtain a further ordered representation of a group of at least 
a subset of the video sequences forming the output media product, the video 
sequences represented by an image with the highest probability of containing human 
faces derived from the respective video sequences. One would have been motivated to 
make such a combination because a way for the system to select a more relevant, user 
friendly representative image would have been obtained, as taught by Dufaux. 

Snook fails to explicitly teach weighting frames according to the size of detected 
human faces derived from the respective video sequences. Dufaux teaches the 
detecting means is operable to detect a probability of a human face being present in 
each field or frame of the video sequences (Fig. 5) and to weight at least some of the 
detected probability levels in dependence on the size of the detected face (Col. 12, lines 
4 et seq.). It would have been obvious to one of ordinary skill in the art, having the 
teachings of Snook and Dufaux before him at the time the invention was made, to 
modify displaying video sequences, represented by a keyframe thumbnail, that may be 
for selected by a user to define a set of one or more video sequences and a further 
ordered representation of a group of at least a subset of the video sequences forming 
the output media product taught by Snook to include the weighting frames according to 
the size of detected human faces derived from the respective video sequences of 
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Dufaux, in order to obtain a further ordered representation of a group of at least a 
subset of the video sequences forming the output media product, the video sequences 
represented by an image with the highest probability of containing a particular sized 
human faces derived from the respective video sequences. One would have been 
motivated to make such a combination because a way for the system to select a more 
accurate face detection method for keyframe selection would have been obtained, as 
taught by Dufaux. 

As in Claim 2, Snook teaches the set of one or more of the video sequences is 
an ordered edited set forming an output media product (Col. 2, lines 62-64). 

As in Claim 3, the above combination of Snook and Dufaux teach the system of 
Claim 1 as rejected supra. Snook further teaches a further ordered representation of a 
group of at least a subset of the video sequences forming the output media product 
(Snook, Col. 8, lines 2-3). Snook fails to explicitly teach images representing human 
faces derived from the respective video sequences in the group. Dufaux teaches 
detected human faces represent their respective video sequences (Dufaux, Col. 4, lines 
4-7). It would have been obvious to one of ordinary skill in the art, having the teachings 
of Snook and Dufaux before him at the time the invention was made, to modify 
displaying video sequences, represented by a keyframe thumbnail, that may be for 
selected by a user to define a set of one or more video sequences and a further ordered 
representation of a group of at least a subset of the video sequences forming the output 
media product taught by Snook to include the means for detecting human faces in the 
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candidate video sequences and assigning them as the key frame representations of the 
candidate video sequences of Dufaux, in order to obtain a further ordered 
representation of a group of at least a subset of the video sequences forming the output 
media product, the video sequences represented by an image representing human 
faces derived from the respective video sequences. One would have been motivated to 
make such a combination because a way for the system to select a more relevant, user 
friendly representative image would have been obtained, as taught by Dufaux. 

As in Claim 4, Snook teaches the ordered representation is a timeline 
representation ("video timeline", Col. 6, line 29), providing an ordered representation of 
the group of video sequences forming the output media product along a generally 
rectilinear path on the display screen (Fig. 6A and corresponding text). 

As in Claim 5, Snook teaches which the ordered representation may be scaled 
so as to vary the proportion of the video sequences forming the output media product 
which are currently displayed in the ordered representation (Fig. 6A ref. 625 and 
corresponding text). 

As in Claim 6, Snook fails to explicitly teach detecting human faces derived from 
the respective video sequences and displaying the one with the highest probability of 
containing a face in the group. Dufaux teaches the detecting means is operable to 
detect a probability of a human face being present in each field or frame of the video 
sequences (Col. 1, lines 55-57); each displayed representation of a candidate video 
sequence including one or more images representing human faces which have the 
highest probability levels amongst the respective video sequences (Col. 12, line 4 et 
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seq.). It would have been obvious to one of ordinary skill in the art, having the teachings 
of Snook and Dufaux before him at the time the invention was made, to modify 
displaying video sequences, represented by a keyframe thumbnail, that may be for 
selected by a user to define a set of one or more video sequences and a further ordered 
representation of a group of at least a subset of the video sequences forming the output 
media product taught by Snook to include the detecting human faces derived from the 
respective video sequences and displaying the one with the highest probability of 
containing a face in the group of Dufaux, in order to obtain a further ordered 
representation of a group of at least a subset of the video sequences forming the output 
media product, the video sequences represented by an image with the highest 
probability of containing human faces derived from the respective video sequences. 
One would have been motivated to make such a combination because a way for the 
system to select a more relevant, user friendly representative image would have been 
obtained, as taught by Dufaux. 

As in Claim 8, Snook fails to explicitly teach weight the probability levels so that 
detected faces closer in size to a desired representation size are more likely to be 
selected to form a displayed representation. Dufaux teaches the detecting means is 
operable to detect a probability of a human face being present in each field or frame of 
the video sequences (Fig. 5) and to weight at least some of the detected probability 
levels in dependence on the size of the detected face (Col. 12, lines 4 et seq.); each 
displayed representation of a candidate video sequence including one or more images 
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representing human faces which have the highest weighted probability levels amongst 
the respective video sequences (Col. 12, line 4 et seq.). Dufaux further teaches the 
detecting means weights the probability levels so that detected faces closer in size to a 
desired representation size are more likely to be selected to form a displayed 
representation (Col. 2, lines 12-25). It would have been obvious to one of ordinary skill 
in the art, having the teachings of Snook and Dufaux before him at the time the 
invention was made, to modify displaying video sequences, represented by a keyframe 
thumbnail, that may be for selected by a user to define a set of one or more video 
sequences and a further ordered representation of a group of at least a subset of the 
video sequences forming the output media product taught by Snook to include the 
weight the probability levels so that detected faces closer in size to a desired 
representation size are more likely to be selected to form a displayed representation 
Dufaux, in order to obtain a further ordered representation of a group of at least a 
subset of the video sequences forming the output media product, the video sequences 
represented by an keyframe image determined according to weight, weight the 
probability levels so that detected faces closer in size to a desired representation size 
are more likely to be selected to form a displayed representation. One would have been 
motivated to make such a combination because a way for the system to select a more 
accurate facial detection method for keyframe selection would have been obtained, as 
taught by Dufaux. 
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As in Claim 9, Snook fails to explicitly teach the detecting means applies the 
weighting over a subset of the fields or frames of a video sequence. Dufaux teaches 
the detecting means applies the weighting over a subset of the fields or frames of a 
video sequence (Col. 2, lines 20-24). It would have been obvious to one of ordinary skill 
in the art, having the teachings of Snook and Dufaux before him at the time the 
invention was made, to modify displaying video sequences, represented by a keyframe 
thumbnail, that may be for selected by a user to define a set of one or more video 
sequences and a further ordered representation of a group of at least a subset of the 
video sequences forming the output media product taught by Snook to include the 
weighting frames according to the size of detected human faces derived from the 
respective video sequences and displaying the one with the highest probability of 
containing a face in the group of Dufaux, in order to obtain a further ordered 
representation of a group of at least a subset of the video sequences forming the output 
media product, the video sequences represented by a detected keyframe image, the 
detecting means is operable to apply the weighting over a subset of the fields or frames 
of a video sequence. One would have been motivated to make such a combination 
because a way for the system to select a more accurate face detection method for 
keyframe selection would have been obtained, as taught by Dufaux. 

As in Claims 10-11, Snook fails to explicitly teach the weighting over frames of a 
video sequence. Dufaux teaches the detecting means applies the weighting over 
frames of a video sequence (Col. 2, lines 20-24). It would have been obvious to one of 
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ordinary skill in the art, having the teachings of Snook and Dufaux before him at the 
time the invention was made, to modify displaying video sequences, represented by a 
keyframe thumbnail, that may be for selected by a user to define a set of one or more 
video sequences and a further ordered representation of a group of at least a subset of 
the video sequences forming the output media product taught by Snook to include the 
detecting means applies the weighting over frames of a video sequence of Dufaux, in 
order to obtain a further ordered representation of a group of at least a subset of the 
video sequences forming the output media product, the video sequences represented 
by a detected keyframe image, the detecting means applies the weighting over frames 
of a video sequence. One would have been motivated to make such a combination 
because a way for the system to select a more accurate face detection method for 
keyframe selection would have been obtained, as taught by Dufaux. 

As in Claim 12, Snook teaches selection of a displayed representation by the 
user control causes the display of the corresponding video sequence (Col. 3, lines 33- 
37). 

As in Claim 14, Snook fails to explicitly teach indicating that faces detected in two 
or more respective ones of the candidate video sequences represent the same person's 
face. Dufaux teaches a user control means indicates that faces detected in two or more 
of the candidate video sequences represent the same person's face (Fig. 5 and 
corresponding text). It would have been obvious to one of ordinary skill in the art, 
having the teachings of Snook and Dufaux before him at the time the invention was 
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made, to modify displaying video sequences, represented by a keyframe thumbnail, that 
may be for selected by a user to define a set of one or more video sequences and a 
further ordered representation of a group of at least a subset of the video sequences 
forming the output media product taught by Snook to include the indicating that faces 
detected in two or more respective ones of the candidate video sequences represent 
the same person's face of Dufaux, in order to obtain a further ordered representation of 
a group of at least a subset of the video sequences forming the output media product, 
the video sequences represented by a detected keyframe image with detected faces 
and indicating that faces detected in two or more respective ones of the candidate video 
sequences represent the same person's face. One would have been motivated to make 
such a combination because a way for the system to select a more accurate face 
detection method for keyframe selection would have been obtained, as taught by 
Dufaux. 

As in Claim 16, Snook teaches computer readable storage medium (Fig. 2 and 
corresponding text) encoded with a computer readable program configured to cause an 
information processing apparatus to execute the method of Claim 15 (See Claim 15 
rejection supra). 

As in Claim 20, Snook teaches a media handling system in which candidate 
video sequences are displayed on a display screen in schematic form for selection by a 
user (Fig. 3 and corresponding text), the system comprising: a display screen 
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configured to display representations of the candidate video sequences for selection by 
a user (ref. 120), each representation including one or more images derived from the 
respective video sequences (Col. 2, lines 64-66); and a user control for defining a set of 
one or more of the video sequences (Col. 2, line 55-Col. 3, line 1). While Snook 
teaches displaying video sequences, represented by a keyframe thumbnail, that may be 
for selected by a user to define a set of one or more video sequences, they fail to show 
the detector to detect human faces in the candidate video sequences to represent the 
candidate video sequences as a keyframe as recited in the claims. In the same field of 
the invention, Dufaux teaches a media handling system similar to that of Snook. In 
addition, Dufaux further teaches detector to detect human faces in the candidate video 
sequences (Fig. 5, ref. 500 and corresponding text); and assigning them as the key 
frame representations of the candidate video sequences (Col. 4, lines 4-7). It would 
have been obvious to one of ordinary skill in the art, having the teachings of Snook and 
Dufaux before him at the time the invention was made, to modify displaying video 
sequences, represented by a keyframe thumbnail, that may be for selected by a user to 
define a set of one or more video sequences taught by Snook to include the detector to 
detect human faces in the candidate video sequences and assigning them as the key 
frame representations of the candidate video sequences of Dufaux, in order to obtain a 
detector to detect human faces in candidate video sequences, displaying video 
sequences, represented by an image representing human faces derived from the 
respective video sequences, that may be for selected by a user to define a set of one or 
more video sequences. One would have been motivated to make such a combination 
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because a way for the system to select a more relevant, user friendly representative 
image would have been obtained, as taught by Dufaux. 

Snook fails to explicitly teach detecting human faces derived from the respective 
video sequences and displaying the one with the highest probability of containing a face 
in the group. Dufaux teaches the detecting means is operable to detect a probability of 
a human face being present in each field or frame of the video sequences (Col. 1 , lines 
55-57); each displayed representation of a candidate video sequence including one or 
more images representing human faces which have the highest probability levels 
amongst the respective video sequences (Col. 12, line 4 et seq.). It would have been 
obvious to one of ordinary skill in the art, having the teachings of Snook and Dufaux 
before him at the time the invention was made, to modify displaying video sequences, 
represented by a keyframe thumbnail, that may be for selected by a user to define a set 
of one or more video sequences and a further ordered representation of a group of at 
least a subset of the video sequences forming the output media product taught by 
Snook to include the detecting human faces derived from the respective video 
sequences and displaying the one with the highest probability of containing a face in the 
group of Dufaux, in order to obtain a further ordered representation of a group of at least 
a subset of the video sequences forming the output media product, the video 
sequences represented by an image with the highest probability of containing human 
faces derived from the respective video sequences. One would have been motivated to 
make such a combination because a way for the system to select a more relevant, user 
friendly representative image would have been obtained, as taught by Dufaux. 
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Snook fails to explicitly teach weighting frames according to the size of detected 
human faces derived from the respective video sequences. Dufaux teaches the 
detecting means is operable to detect a probability of a human face being present in 
each field or frame of the video sequences (Fig. 5) and to weight at least some of the 
detected probability levels in dependence on the size of the detected face (Col. 12, lines 
4 et seq.). It would have been obvious to one of ordinary skill in the art, having the 
teachings of Snook and Dufaux before him at the time the invention was made, to 
modify displaying video sequences, represented by a keyframe thumbnail, that may be 
for selected by a user to define a set of one or more video sequences and a further 
ordered representation of a group of at least a subset of the video sequences forming 
the output media product taught by Snook to include the weighting frames according to 
the size of detected human faces derived from the respective video sequences of 
Dufaux, in order to obtain a further ordered representation of a group of at least a 
subset of the video sequences forming the output media product, the video sequences 
represented by an image with the highest probability of containing a particular sized 
human faces derived from the respective video sequences. One would have been 
motivated to make such a combination because a way for the system to select a more 
accurate face detection method for keyframe selection would have been obtained, as 
taught by Dufaux. 
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6. Claim 13 is rejected under 35 U.S.C. 103(a) as being unpatentable over Snook, 
US Patent 6400378, Dufaux, US Patent 6711587 and further in view of Trivedi et al., US 
Patent Application Publication 2006/0187305, hereinafter Trivedi. 

Snook and Dufaux teach means for detecting human faces in candidate video 
sequences, displaying video sequences, represented by an image representing human 
faces derived from the respective video sequences, that may be for selected by a user 
to define a set of one or more video sequences (Claim 1 rejection supra). While Snook 
and Dufaux teach displaying video sequences, represented by a keyframe thumbnail, 
that may be for selected by a user to define a set of one or more video sequences, 
detecting human faces in the candidate video sequences to represent the candidate 
video sequences as a keyframe, they fail to show the candidate video sequences are 
selected from a video sequence captured by a surveillance camera as recited in the 
claims. In the same field of the invention, Trivedi teaches a media handling system and 
facial detection similar to that of Snook and Dufaux. In addition, Trivedi further teaches 
in which the candidate video sequences are selected from a video sequence captured 
by a surveillance camera (Par. 0038). It would have been obvious to one of ordinary 
skill in the art, having the teachings of Snook, Dufaux before him at the time the 
invention was made, to modify displaying video sequences, represented by a keyframe 
thumbnail, that may be for selected by a user to define a set of one or more video 
sequences, detecting human faces in the candidate video sequences and assigning 
them as the key frame representations of the candidate video sequences taught by 
Snook and Dufaux to include the candidate video sequences are selected from a video 
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sequence captured by a surveillance camera of Trivedi, in order to obtain means for 
detecting human faces in candidate video sequences captured by a surveillance 
camera, displaying video sequences, represented by an image representing human 
faces derived from the respective video sequences, that may be for selected by a user 
to define a set of one or more video sequences. One would have been motivated to 
make such a combination because a way for police and security guards to single out 
people from video surveillance would have been obtained, as taught by Trivedi. 



Response to Arguments 

Applicant's arguments filed 1 1/3/08 have been fully considered but they are not 
persuasive. 

The applicant states "that a face picture stamp output purely based on face 
probability does not always give the best quality picture stamp" and goes on to suggest 
that the detected probability levels are based on more than probability of a face in a 
frame based on the size of the detected face (page 9 of the remarks submitted 1 1/3/08), 
however the claims state that the probability level is at least limited by the size of the 
face, and does not state any other factors that determine the probability level. 

The examiner agrees that Snook fails to teach detecting human faces in the 
candidate video sequences for detecting the probability of a human face being present. 

The applicant states that Dufaux "is silent regarding a user control for defining a 
set of one or more video sequences" (page 1 0, lines 1 3-1 4), however this limitation has 
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already been taught by Snook as mapped above: "a user control means for defining a 
set of one or more of the video sequences (Col. 2, line 55-Col. 3, line 1)". 

In response to the applicant's argument that "Dufaux is silent regarding weighting 
a detected probability level depending on a size of a detected face" (page 10, lines 21- 
22) the examiner disagrees. Dufaux teaches giving weight, referred to as a weighting 
factor (Col. 2, line 24) to determine the shot score, according to different limitations 
including the skin pixels (Col. 2, line 22). This is further illustrated by Dufaux in Col. 6, 
beginning on line 41 , when they begin to discuss the process of determining if the frame 
has people in it. Frames that have smaller faces are not given as much weight because 
they "are not likely to be representative of the video" as a whole. 

The prior art made of record on form PTO-892 and not relied upon is considered 
pertinent to applicant's disclosure. Applicant is required under 37 C.F.R. § 1 .1 1 1 (c) to 
consider these references fully when responding to this action. The documents cited 
therein teach similar video editing and keyframe selection techniques as well as 
methods for detecting human faces within video. 
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Conclusion 

Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Sara M. Hanne whose telephone number is (571) 272- 
4135. The examiner can normally be reached on M-F 7:30am-4:00pm, off on 
alternating Fridays. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, WEILUN LO can be reached on (571) 272-4847. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 

Patent Application Information Retrieval (PAIR) system. Status information for 

published applications may be obtained from either Private PAIR or Public PAIR. 

Status information for unpublished applications is available through Private PAIR only. 

For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 

you have questions on access to the Private PAIR system, contact the Electronic 

Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 

USPTO Customer Service Representative or access to the automated information 

system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Sara M Hanne/ 
Examiner, Art Unit 2179 



